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A Technology in Texas Public Schools 1998 Survey instrument was integrated into a 
web-based response system enabling the instrument to be accessed, completed, submitted 
and instantaneously analyzed over the Internet. A mark-sense or optical scan paper 
version of the instrument was also developed for mail-out distribution to each school 
district in the state. The decision to provide two response options to school 
administrators for this survey provided the research question for this investigation. That 
is, “Is there a difference in survey responses submitted by Internet or mail?” The 
findings revealed no observed differences in responses whether the survey was submitted 
electronically or by mail. Other issues of cost, time and human networks are discussed 
and in general, the authors support the use of the Internet for conducting surveys with 
large samples. 
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BUILDING A CASE FOR CONDUCTING TECHNOLOGY SURVEYS ONLINE 



Jon Denton and Aden Strader 

Instructional applications of computer technology have been proposed and tested over the 
past 30 years. To illustrate, in 1969 Lawrence Stolurow proposed standards for computer 
based instructional systems designed for the individual learner (Stolurow, 1969). These 
standards were subsequently applied in secondary school physics classes by the senior 
author and a colleague in separate investigations on computer managed instruction (note 
1). These instructional systems, tailored for individual learners have evolved into 
intelligent tutoring systems that today remain on the horizon of instructional applications 
of computer technology for public school classrooms. More often, early classroom 
applications of computers emphasized tutorials, drill and practice and computer managed 
learning activities (Kulik, 1984). 

High profile reports including one by the President’s Committee of Advisors on Science 
and Technology (1997) suggest that computer technology may have the potential to 
influence the future of elementary and secondary education. Bringing this goal into focus 
on student learning and the classroom, Bransford, Brown and Cocking (1999) conclude 
after reviewing numerous efforts currently underway, that: 

...technology-based tools can enhance student performance when they are 
integrated into the curriculum and used in accordance with knowledge 
about learning... But the mere existence of these tools in the classroom 
provides no guarantee that student learning will improve; they have to be a 
part of a coherent education approach, (page 204). 

The prospect of computer technology shaping tomorrow’s schools to enable learners to 
succeed academically is certainly more attainable given today’s Internet networks and 
powerful multimedia- networked workstations than 30 years ago. Yet issues associated 
with policy positions about technology integration, technology infrastructure, technology 
support provided by school districts, professional staff development on technology 
applications and use of technology by teachers and their students have become important 
benchmarks to mark progress with the integration of technology into our classrooms. 

Policy Issues and Technology Infrastructure: In 1995, the state legislature passed three 
acts that have accelerated the development of technology integration in schools. 
Significant efforts to build technology infrastructure in the state are evident with over 
1,000 grants approaching $300,000,000 in total volume being awarded by state agencies. 
These awards have targeted rural communities serving children often from low-income 
homes. Given this intensive effort, it is important to consider how these resources have 
been used to put support systems in place and what professional development activities 
on these technologies are occurring in the schools. 

Staff Development and Technology Support: With schools becoming connected and 
educators participating in staff development activities about integrating multimedia into 
classroom instruction, much momentum will be lost if integration is discussed but not 
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implemented. Few states invest adequately in either pre-service or in-service technology 
professional development for educators. As a result, most teachers have little direct 
experience in observing and learning about the wide range of computer- 
telecommunications applications for classrooms. An Education Commission of the 
States document (ECS, 1998) states that only 15 percent of the K-12 teachers in the 
nation have received as much as 9 hours of training in technology. Further, this report 
notes that the average school district expenditures for technology devoted to teacher 
training is reported to be 6% while the recommended level is 30%. 

Literature on technology staff development indicates that best practice takes place in the 
school when teachers experience the need to change their classroom practices. Optimal 
professional development opportunities also center around specific projects and take into 
account the varied levels of experience of educators with technology that occur across an 
extended period of time (Clark & Denton, 1998). 

Use of Technology by Teachers and Their Students: Baseline data collected in 1996 
across the state indicate that very few teachers and students accessed the Internet for 
classroom instruction. If the technology infrastructure changes and staff development 
programs have been successful, dramatic increases in use by teachers and students will 
have occurred. These data on level of technology applications by teachers and their 
students are invaluable to policy makers in determining next steps for funding technology 
integration in schools. 



Context 

In 1996, a mailout survey of the technology infrastructure in all public schools in the state 
was conducted. Over (82)% or (856) of the 1,043 school districts in the state participated 
in that survey effort. Findings from that effort included: 

• Over half (57%) of the school districts reported having 2-6 computers (386Mhz or 
above) per classroom; 

• Twenty percent of the reporting districts reported no staff development on technology 
was provided in their schools; 

• Over one-third of the school districts reported on-going technical assistance being 
available in their schools; 

• Over 80% of the responding districts planned to spend 10 cents of each dollar 
budgeted for technology on staff development activities over the following three 
years; 

• Nearly half of the responding school districts indicated that the district’s technology 
plan was revised annually; 

• Eighty-seven percent of the respondents reported technology hardware expenditures 
ranged from 0 to 2 cents per dollar of the district budget. 

The results of this 1996 Technology Infrastructure Survey are available at 
http://eEducation.tamu.edu . This novel site provides an electronic file and associated 
software, Web Survey Builder that enables the data to be partitioned and analyzed with 
respect to different geographic and school size classifications enabling customized 
reports for each reader. Anecdotal evidence indicates this site has been frequently 
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accessed and used in developing proposals for technology support by schools across the 
state. Further, this survey effort was referenced in a state agency communication as a 
source of information that was used in determining how technology funds were 
distributed to school districts across the state. Given the level of use by school districts 
and state agencies with the 1996 Technology Infrastructure Survey, a decision was 
readily made to undertake another technology survey. 

The capabilities of the Web Survey Builder were further applied by loading the Levels 
and Use of Technology in Texas Public Schools 1998 Survey instrument into this 
system enabling the instrument to be accessed, completed, submitted and instantaneously 
analyzed over the Internet. A mark-sense or optical scan paper version of the instrument 
was also developed by National Computer Systems (NCS) for mailout distribution to 
each school district in the state. The decision to provide two response options to school 
administrators for this survey provided the basis for this investigation. 

Research Questions 

The following research questions were posed to guide this investigation. 

Is there a difference in survey responses submitted by Internet or mail with respect to: 

1. policy issues about technology integration? 

2. technology support provided by the school district? 

3. technology infrastructure of school district? 

4. uses of technology by teachers and students? 

5. staff development on technology integration? 



These questions are important pragmatically for a number of reasons. First, because 
different response options are available, the Internet response option selected may signal 
school districts who are early adopters and more aligned with technology integration. 
Although this potential phenomenon would not pose a problem for a single 
administration, it could affect interpretations of data across surveys conducted at different 
times. If the response patterns are different, the different collections procedures would be 
a limitation in making comparisons across time, (i.e., 1996 Survey (mailout option only), 
1998 survey (Internet and mailout options) with a survey potentially being conducted 
electronically in 2000). Second, the direct costs associated with a mailout survey are 
considerably higher than an electronic survey conducted over the Internet, once the 
Internet protocol has been developed and tested. Third, the time necessary to conduct the 
survey may be decreased if school administrators actually use the Internet effectively in 
conducting their professional duties. 
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Method 



Instrument Development: An initial draft of the survey instrument was developed based 
on the 1996 instrument and critiqued by technology representatives from: the state 
education agency; a regional educational service center; a telecommunication company; a 
school administrator professional association; the T.H.E. Journal; and SCR*TEC-TX 
staff (note 2). The instrument subsequently underwent several revisions that incorporated 
suggestions from staffs of state agencies and legislators resulting in the final version that 
contained 37 items clustered under five headings: policy (5 items); district’s technology 
support (5 items); district’s current technology infrastructure (10 items); use of 
technology (10 items); staff development related to technology (7 items). Although the 
distribution of the survey was targeted to the superintendent of schools, directions at the 
beginning of the instrument encouraged the superintendent to direct the instrument to the 
district’s technology director for completion and remittance. Approval of the instrument 
and data collection process by the university Institutional Review Board for Human 
Subjects in Research was attained in late May 1998. The instrument may be viewed at 
http://eEducation.tamu.edu . 



Data Collection 

Data collection began with the initial survey mail-out occurring on August 1 1, 1998 to all 
1,043 school districts in the state. An initial mail-out packet was provided to the U.S. 
Postal Service for first class delivery. This packet contained a personalized cover letter 
from the Executive Director of the school administrator’s association to the 
superintendent, a distinctive information sheet on accessing the electronic version of the 
survey instrument, a mark sense survey instrument and a return mailer. During the 
following eight weeks 468 completed surveys were submitted representing a 44.9% 
return. 

A second mail-out consisting of a personalized reminder cover letter from the Executive 
Director to the superintendent, a mark sense survey instrument, return mailer, and a 
descriptive statistical summary of the initial 450 responses for each survey item was 
mailed with first-class service to non-responding school districts on October 12, 1998. 
Five weeks later, 663 completed surveys had been submitted. 

A reminder from the school administrator’s association was made to nineteen non- 
participating districts on November 30, 1998, and reminder telephone calls to the 
remaining non-participating districts began November 30, 1998 by SCR*TEC-TX staff. 
Offers to send a third instrument (137 districts mailed another copy) as well as 
encouragement to submit survey information electronically (website addresses provided 
to 99 districts) and 16 instruments were faxed to districts as a result of these reminders. 
On Tuesday, January 5, 1999 the closing date for receipt of surveys, slightly less than 
four months after the data collection began, 789 completed surveys had been submitted. 




6 



4 



Because two response options (electronic and mark sense) to the survey were provided to 
respondents, the type of return was recorded as completed surveys were received. Table 
1 provides a summary of the electronic and mailed surveys. 



Table 1: 1998 Public School Technology Survey Log 





Mail In 
Responses 


Internet 

Responses 


Cumulative Total 


First Mail-Out 


291 


177 


468 


Second Mail-Out 


165 


34 


667 


Third Mail-Out 


50 


72 


789 



During the initial two weeks of data collection, the electronic submissions outnumbered 
the mailed submissions 2:1. After this initial period, the mailed responses increased 
resulting in the ratio of 37:63 of electronic to mailed responses by October 12, 1998. 
Due to an oversight, a distinctive information sheet on accessing the electronic version of 
the survey instrument was not included in the second mail-out, yet some school officials 
noted the URL for the electronic version in the directions of the mark sense instrument 
and submitted the completed survey electronically. At the close of data collection a total 
of 789 surveys were submitted from 75.6 % of the school districts in the state. Thirty-six 
percent of the responses were submitted electronically and the remainder was mailed for 
machine scoring at the university’s Office of Tests and Measurements. 

Data Analysis 

Data received through electronic submissions were verified with respect to the district 
name and/or district-county identification number. These data were then organized into 
Excel files. As completed mail-in surveys were received; they were verified with respect 
to district name and identification number and checked with respect to readability by the 
machine scoring equipment. Batches of 40-50 mailed surveys were delivered to Tests 
and Measurements Services for processing at a time. Resulting electronic data files from 
this process were then concatenated with the electronically submitted data for processing 
with the Web Survey Builder. As noted earlier, this software enables data to be 
partitioned and analyzed with respect to different geographic and school size 
classifications enabling customized reports for each reader. Simple cumulative 
summaries (frequencies and percentages) are provided by this software, but additional 
statistical analysis using the SPSS statistical package were conducted to answer the 
research questions posed in this investigation. For most items the data were categorized 
by item number from the survey instrument and response option (print or electronic) into 
contingency tables and tested with the % 2 goodness of fit statistic for differences in 
response patterns. The x 2 goodness of fit statistic tests the degree to which differences 
between response patterns can be assigned to chance. 




7 



5 



Findings 



The following data summaries have been prepared to address the research questions that 
guided this effort. Except for the analyses of all items dealing with policy (question 1), 
samples of items were drawn to represent the analyses for each of the remaining sections. 
However, statistical tests were conducted on all 37-survey items with respect to whether 
the survey was submitted electronically or by mail. The results were very consistent and 
uniform; that is, no statistical differences were noted with respect to whether the survey 
item was completed on a computer screen or on paper. 



Question 1: Is there a difference in survey responses submitted by Internet or mail 
with respect to policy issues about technology integration? 

Five items of the instrument were grouped under policy issues, although one of the items 
contained six parts yielding a group of 10 items. Data from these items were analyzed 
together. Table 2 presents histograms for each of the items. Histogram plots represent 
positive responses for all items. The five items addressed whether the school district had : 



Item 1. benefited from state legislation on technology infrastructure 
Item 2. expressed interest in electronic delivery of class materials 
Item 3. considered free or low-cost Internet access to be a state priority 
Item 4. applied for federal telecommunications rebate 
Item 5a. interest in technology audits 
Item 5b. interest in technology use plans 

Item 5c. interest in technology infrastructure plans with projected costs 
Item 5d. interest in establishing technology consortia 
Item 5e. interest in staff development on technology integration 
Item 5f. interest in grant writing workshops for technology funding 



Table 2: Histograms of Mail and Internet Responses on policy issues 



Policy and Infiastmctuie (ChiSquare=7.16, p=0.62) 
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The difference in response patterns across all items in this analysis were not statistically 
significant (x 2 = 7.16, p=0.62). This finding supports comparing survey data sets 
collected either by mail, Internet or both mail and Internet with one another because the 
observed differences due to the type of response technique can be explained by chance. 
As anticipated, items about technology audits, technology planning, and technology 
consortia did not generate the level of affirmation as items addressing state or federal 
financial support, grant writing support, technology integration into instruction and staff 
development on technology integration. 

Question 2. Is there a difference in survey responses submitted by Internet or mail 
with respect to technology support provided by the school district? 

Five items of the instrument (items 6-10) were grouped under technology support. Items 
6 and 7 requested information on expenditures on technology for equipment and staff 
development yielding interval data. For item 6 (total annual technology expenditures) the 
medians are $167,000 for the Internet group (n=219) and $150,000 for the Mail group 
(n=467). This yielded a p-value using Wilcoxen W of 0.145. For item 7 (total annual 
staff development expenditures) the medians for both groups was $5,000 (n=226 for 
Internet group and n=459 for Mail-In group) yielding a p-value of 0.727. Both are 
insignificant effects. 

Question 3. Is there a difference in survey responses submitted by Internet or mail 
with respect to the technology infrastructure provided by the school district? 

Ten items of the instrument (items 1 1-20) were grouped under technology infrastructure 
with many of the items containing multiple parts yielding multiple comparisons. As 
examples for this group, consider “11a. Elementary Campus Computer-to-Student Ratio,” 
“12b. Average Number of computers per middle school classroom,” and “13c. Ratio of 
high school campuses with Internet access.” Table 3 illustrates that for these three 
questions the response distributions of the two groups are very similar. Table 4 shows 
the x 2 values for items 11-13. None of these items show statistical significant differences 
between the Mail-In and Internet groups. 
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Table 3. Technology infrastructure response comparisons 



Elementary Corrputer- 
Studant Ratio 




□ >1:25 

■ 1:25 

□ 1:15 

□ 1:10 

□ 1:5 

□ 1:3 

■ 1:1 



Middle School 
Computers per 
Classroom 




High School Campuses 
w 1 1nternet Access 



■ >10 
□ 6-10 

□ 2-5 

□ 1 
□ 0 
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Mail-In Internet 



>=80% 

<70% 

<50% 

<40% 

<30% 

< 20 % 
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0 



Table 4. % 2 values for items 11-13 





Elementary 


Mid 


die 


High School 




x 1 


P 


x 2 


P 


x 2 


P 


1 1 . Campus Computer to Student Ratio 


9.32 


0.10 


8.33 


0.14 


5.89 


0.32 


12. Computers per classroom 


4.57 


0.21 


5.39 


0.15 


2.56 


0.46 


13. Campuses with Internet Access 


6.51 


0.48 


7.74 


0.36 


6.59 


0.47 



Question 4. Is there a difference in survey responses submitted by Internet or mail 
with respect to use of technology by teachers and students in the school district? 

“Use of technology by teachers and students” consists often items within the instrument 
(items 21-30). Many of these items containing multiple parts yielding comparisons. 

Items 21, 22, 29, and 30 illustrate this section. Item 21 compares the number of teachers 
using the Internet for classroom instruction and the data are pictured in Table 5 (% 2 = 
10.02, p=0.53). Item 22 compares the frequency of classroom use of the Internet. This 
data are shown in Table 6 (% 2 = 8.44, p=0.21). Table 7 depicts the data from items 29 and 
30 that compare students’ access to computers and the Internet at home (29. % 2 = 6.40, 
p=0.09; 30. x 2 = 8.27, p=0.08). 
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Table 5. Comparison of classroom use of the Internet 



21. Internet used inClassroom as part of Instruction (CHSqr=10.02,pp0.53) 
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Table 6. Histogram for Item 22. 
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Table 7. Histogram comparing Items 29 and 30. 



29-30. Students' Access at Home (Ch5qr^).098,p>.99) 




0 - 25 % 26 - 50 % 51 - 75 % 76 - 100 % 

Access to a computer 



















za 




1 


. ^~1 . 



■ Mail-In 
□ Internet 



1 - 25 % 26 - 50 % 51 - 75 % 76 - 100 % 

Access to the Internet 



Question 5. Is there a difference in survey responses submitted by Internet or mail 
with respect to staff development related to technology in the school district? 

Seven items (items 31-37) were grouped under the final section of the instrument, staff 
development related to technology. For this group we use item 3 1 to illustrate the point 
of this paper. Item 3 1 compares the types of professional development provided by 
school districts for their teachers. The response patterns are portrayed in Table 8. These 
values were analyzed using the % 2 test with the results given by % 2 = 5.82 and p=0.67. 

The x 2 statistics for the remaining items are summarized in Table 9. 



For the items in this group 

and coordinators (Chi Square = .004, p> .99), effectiveness/constraints for technology 
staff development (Chi Square = .245, p> .99), and technology planning frequency (Chi 
Square = .038, p> .99) were not statistically different with observed differences being 
attributed to chance. In 1998, common topics for staff development included Internet 
access, content-focused applications, and ethical/legal issues associated with technology 
integration into the classroom. While a number of districts reported their staff 
development programs were effective, other districts reported time and funding to be the 
major constraints in providing effective technology staff development programs. Also, 
technology plans appear to be revised often, suggesting technology integration is 
dynamic and rapidly evolving into the curricular programs of these school districts. None 
of these analyses resulted in statistically different responses between the groups. 
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Table 8. Response patterns for Item 31 regarding professional development of faculty. 




Table 9. Summary statistics for technology professional development questions 





x 2 


P 


32. Number of professional development sessions annually available 
to teachers. 


7.32 


0.12 


33. Types of professional development provided to technology 
coordinators 


4.13 


0.84 


34. Frequency of technology plan revisions 


7.09 


0.13 


35. Who provides technology staff development? 


3.03 


0.55 


36. Is current technology staff development sufficient? 


0.14 


0.71 


37. Stumbling blocks for more technology staff development 


2.52 


0.47 



Conclusions and Discussion 

The answers to the research questions regarding whether the submission option of mail or 
Internet affected the response patterns are five unequivocal “No”s. All of the statistical 
analyses conducted across the five research questions provided the same response; no 
difference in response patterns whether the data were submitted by mail or the Internet. 

These findings support a move toward conducting future surveys with school districts in 
this state via the Internet. One of the reasons this investigation was conducted was our 
concern that a response bias might occur with early adopters of technology answering the 
questions differently than districts responding by mail. This hunch was initially 
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supported by submission behavior of districts when the electronic submissions 
outnumbered the mailed submissions dramatically over the initial two weeks of data 
collection. The consistent “no significant difference” findings across the entire survey 
also quieted our concerns about whether we could compare the 1998 survey results with 
survey data obtained at different times with only one submission option (mail or 
Internet). In fact, a companion paper (Denton, Davis, Strader, Jessup & Jolly, 1999) has 
been prepared that examines technology infrastructure changes between 1996 and 1998 
across the state. Had these analyses revealed a number of items with significant 
differences due to the type of submission, a source of error variance would have been 
identified that would have reduced the significance of that paper for state policy makers. 

The consistent findings of this investigation also support using the Internet to gather 
future needs and trends data from school districts across the state. Pragmatically, the cost 
of conducting a statewide survey is potentially lower using the Internet as the delivery 
system compared to mail delivery. In the case of this effort, each electronically 
submitted survey saved over a dollar in mailing costs alone. Additional costs for 
machine scoring of the mailed survey added a few additional cents to the mail method of 
data collection as well as the considerable cost of printing the mark-sense paper version 
of the instrument. However, the cost of the computer equipment and telecommunication 
system needed to support the Internet delivery system far offsets the preceding costs 
noted for conducting a survey by mail. Fortunately, the development (and related costs) 
of the Web Survey Builder had already occurred and development issues for the 1998 
Survey required about 40 hours. Also, information gathered from the 1998 survey (see 
Denton, et al., 1999) indicate the telecommunications infrastructure is in place across the 
state to support the use of Internet surveys. 

A related issue is whether the level of response will be maintained if Internet submission 
is used. In 1996, a response ratio of .82 occurred from school districts in the state, while 
in 1998, a response ratio of .756 occurred. Although the response ratio in 1998 was less, 
it still was quite good and the resulting data are thought to represent the status of 
technology in the state’s public schools. It is felt the primary reason for the substantial 
response ratios for these surveys was the influence of the school administrator 
professional association in convincing school administrators to accept responsibility for 
submitting a complete and accurate survey because of their potential impact on state 
policy support for technology. Thus, influential human networks are necessary for 
substantial responses to statewide survey requests. 

Finally, the variable of time and just-in-time information may be important 
considerations in how future statewide surveys are conducted. By using the data 
management capabilities and report production functions of the Web Survey Builder, 
status reports were developed frequently during the data collection phase of the survey 
and shared with state policy groups. These reports provided descriptive quantitative 
summaries on each survey item. Because the database of the Web Survey Builder is 
dynamic, it was updated with each electronic submission. This attribute of the system 
enabled a preliminary report to be developed by the system, printed and included in the 
second mailout to non- respondents. It also enabled the project team to deliver a printed 
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report to all 1,043 school districts within two weeks of the end of data collection phase of 
the survey effort. 

In closing, the findings of this investigation support processes of submitting a survey 
electronically or by mail or using both mail and the Internet as viable options because no 
observed differences in responses occurred due to the delivery of data. Other issues of 
cost, time and human networks were discussed and in general, the authors support the use 
of the Internet in conducting surveys with large samples. 



Note 1 : Two dissertation research efforts occurred that incorporated tenets proposed by 
L. Stolurow. These investigations were: 

Denton, JJ. (1972). A methodological study of a computer managed instructional 
program in high school physics. Unpublished doctoral dissertation, University of 
Missouri, Columbia, MO. 

Paden, J.S. (1970). An experimental study of individualized instruction in high school 
physics using the computer to prescribe activities as a function of selected ideographic 
factors. Unpublished doctoral dissertation, University of Missouri, Columbia, MO. 

Note 2: The South Central Regional Technology in Education Consortia - Texas 
(SCR*TEC-TX) staff consists of six people including the two authors. This staff was 
supported with funds from the U.S. Department of Education to facilitate the integration 
of technology into public school classrooms through September 2000. 
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